Phy-Q as a measure for physical reasoning intelligence

Abstract Humans are well versed in reasoning about the behaviours of physical objects and choosing actions accordingly to accomplish tasks, while this remains a major challenge for artificial intelligence. To facilitate research addressing problem, we propose new testbed that requires an agent reason scenarios take action appropriately. Inspired by knowledge acquired infancy capabilities required robots operate real-world environments, identify 15 essential scenarios. We create wide variety distinct task templates, ensure all templates within same scenario can be solved using one specific strategic rule. By having such design, evaluate two levels generalization, namely local generalization broad generalization. conduct extensive evaluation with human players, learning agents various input types architectures, heuristic different strategies. how intelligence quotient is calculated, define (Phy-Q score) reflects considered. Our shows (1) far below performance, (2) agents, even good ability, struggle learn underlying rules fail generalize broadly. encourage development intelligent reach human-level Phy-Q score.